Searching and Querying Wide-Area Distributed Collections

The rapid proliferation of widely-distributed data and document collections raises the need for wrapper/mediator archi-tectures that can handle the challenges of wide area query processing. Traditional query and search techniques do not scale to large numbers of repositories and cannot cope with the unpredictable performance and (un)availability of access to such repositories. Research at the University of Maryland is aimed at addressing the following challenges: Query planning for wide area networks: We describe Web query optimization techniques that use a Web-Wrapper cost model (WCM) and WebPT-a tool to predict response times from WWW sources. Coping with unexpected delays: Query Scrambling is a reactive query execution scheme that adapts the query plan in response to runtime delays. XJoin is a small footprint, fully pipelinable join operator that automatically adjusts the ow of tuples during query execution. Planning with alternate sources: We investigate strategies for chosing among multiple alternative data sources, and techniques to adjust these decisions when severe delays are encountered. Source publishing and selection: We describe how sources can be published on the WWW using XML, and we investigate source selection using content and quality metadata. Acknowledgements: We are very grateful to Laura Bright and Tao Zhan for their research and programming support.